Visual-Based Emotional Descriptor and Feedback Mechanism for Image Retrieval

A new emotion-based image retrieval method is proposed in this paper. Query emotional descriptors called query color code and query gray code are designed on the basis of human evaluation of 13 emotion pairs (“like-dislike”, “beautiful-ugly”, “naturalunnatural”, “dynamic-static”, “warm-cold”, “gay-sober”, “cheerful-dismal”, “unstablestable”, “light-dark”, “strong-weak”, “gaudy-plain”, “hard-soft”, “heavy-light”) when 30 random patterns with different color, intensity, and dot sizes are presented. For emotion image retrieval, once a query emotion is selected, the associated query color code and query gray code are obtained, and DB color code and DB gray code that capture color, intensity, and dot size are extracted from each database image. Next, a matching process between the two color codes and between the two gray codes is performed to retrieve images with a sensation of the query emotion. The new relevance feedback method proposed here incorporates the human information needs in the retrieval process by dynamically updating relative weights between the query and DB color codes, and among relative weights of an intra query color code. To show the validity of the proposed method, experiments are performed on over 450 images.